AU9884253 



(1 2) PATENT ABSTRACT (1 1 ) Document NO. AU-A-84253/98 
(19) AUSTRALIAN PATENT OFFICE 

(54) Title 

CONTENT INFORMATION GATHERING APPARATUS, SYSTEM AND METHOD 

International Patent Classtfication(s) 
(51)« G06T 005/00 

(21) Application No. : 34253/98 (22) Application Date : 14/09/98 

(30) Priority Data 

(31) Number (32) Date (33) Country _ 

P09186 15/09/97 AU AUSTRALtA 

(43) Publication Date : 15/04/99 

(71) Applicant(s) 

CANON KABUSHIKI KAISHA; CANON INFORMATION SYSTEMS RESEARCH AUSTRALIA PTY LTD 

(72) Inventor(s) 
JOHN RICHARD WINDLE 

(74) Attorney or Agent 

SPRUSON & FERGUSON , GPO Box 3898, SYDNEY NSW 2001 

(57) A content gathering apparatus (100) including content collecting means (101) for O 
gathenng content information and teniplate storage means (103) for storing at least one 
template including a plurality of template instmctions. Krocessing moans (104) are 
used to implement the template instmctions. A user interface (105) displays 
mformation to a user of the apparatus (100). The template instructions, when executed 
provide composition suggestions to the user of the content gathering apparatus (100) in 
relation to enhancing composition of the content information collected, the suggestions 
being provided via the user interface (105). Methods and systems for implementing the ^ 
invention are also disclosed. ^ y?^^ 



S & F Ref: 429470 

AUSTRALIA 
PATENTS ACT 1990 

COMPLETE SPEanCATlON 

FOR A STANDARD PATENT 



ORIGINAL 



Name and Address 

of Applicant: Canon Kabushiki Kaisha 

30-2, Shimomaruko 3-chome 

Ohta-ku 

Tokyo 146 

JAPAN 

Canon Information Systems Research Australia Pty Ltd 

1 Thomas Holt Drive 

North Ryde New South Wales 2113 

AUSTRALIA 



Actual Inventor(s>: John Richard Windle 

Address for Service: Spruson & Ferguson, Patent Attorneys 

Level 33 St Martins Tower, 31 Market Street 
Sydney, New South Wales. 2000, Australia 

Invention Title: Content Information Gathering Apparatus, System and 

Method 

ASSOCIATED PROVISIONAL APPLICATION DETAILS 

[31] Application No(s) C33] Country [32] Application Date 

P09186 AU 15 September 1997 



ihe following statement Is a full description of this Invention, 
Including the best method of performing it known to me/us:- 



CONTENT. ENFORMATION GATHERING APPARATUS, 
SYSTEM AND METHOD 



The present invention relates to content information gathering devices, and. 
more particularly, to a method, system and apparatus to increase the composition 
quality of content information gathered by such devices. 

The invention has been developed primarily for use with video and audio 
^ capture and" recording devices; and-will be described" herein with reference ~t6~ these 
applications. However, it will be appreciated that the invention is not limited to these 
fields. 

Background 

There exists a relatively large number of devices for collecting content 
information such as sounds and images. These devices are frequently used in 
conjunction with a recording device to provide a combined collecting and recording 
apparatus. Examples of such corrbined apparanjs include video cameras, digital still 
cameras, audio tape recorders, 3D image recorders and motion capture systems. 
Alternatively, content collection devices can be used in conjunction with processing 
devices that process the gathered conrent and then pass the processed information to 
another device. Examples of such processing devices include audio or video mixing 
desks, and electronic processors. 

Over time, the faithfulness with which content collection devices have gathered 
content has steadily improved. Whilst physical, optical and mechanical advancements 
have provided much of this improvement, electronic assistance has also contributed 
greatly. 

An example of the improvements in useability offered by electronic 
enhancements is the photographic camera. Early mechanical cameras required the user 
to have a relatively good understanding of the mechanisms of image production on film. 
This included the need for detailed knowledge about lighting, shutter speed, and 
aperture settings for various photograph and film types. Initially, even such relatively 
simple factors as available light required an estimate based on a subjective assessment 
by the experienced user. For these reasons, good technical reproduction on film was 
only obtainable by professionals and knowledgable amateurs, since it required much 
practice a.id experience. 

In contrast, many modern cameras are equipped with electronic instruments 
which measure the available light and automatically adjust aperture and speed settings, 
taking into account the film type. As a result, the technical quality of photographs 
taken with modern cameras is, on average, much higher than that of older mechanical 
cameras, particularly for the relatively inexperienced amateur. 



Whilst the technical reproduction aspects of amateur photography have 
improved over lime, the composition, or arrangement, of subjects within capnired 
frames has not generally improved. Even experienced amateur photographers still take 
shots that leave portions of the subject out of frame (eg, a person's head), take multiple 
pan shots which do not line up and so on. Even seuing aside more subtle aesthetic or 
artistic aspects, the ability of professional photographers to correctly compose a 
photograph means that professionally taken photographs are still, on average, superior 
to those of amateurs,- — _ ^ 

Whilst particularly applicable to photography, the problems of composition are 
relevant to other areas where content information is gathered. An example is the area 
of audio recording. In tlie past, the recording of music was confined to expensive 
studios, and professional producers and engineers capable of correcdy composing a 
number of available channels of sound into a "mix" to produce a master recording. 

However, with rapid advancements in digital technology, relatively small 
operators can now buy and operate equipment that was previously restricted to the 
professional user. Whilst the cheaper equipment is capable of relatively high recording 
qua'ity, often the composition of recorded material into a compler'id recording will still 
be of relatively low quality, due to a lack of experience and knowledge on the part of 
the amateur operator. 

Accordingly, a need exists for a method and apparatus that overcomes one or 
more of the disadvantages associated with quality composition of content information 
collected by content collecting devi.:es. A need exists for adding value, in a 
predetermined manner, to the content information. 

Summary 

In accordance with a first aspect of the present invention, there is provided 
content gathering apparatus comprising: 

content collecting means for gathering content information; 

communication means for uploading to the enhanced gathering apparatus at 

least one template, wherein the template comprises a plurality of template 
instructions; 

processing means for executing the template instructions for providing 
composition instructions for enhancing a composition of the gathered content 
information and/or provide user selection options; 

memory storage means for storing the template and for storing the gathered 
content information; and 

user interface means for displaying the user selection providing option. 

In accordance with a second aspect of the present invention, there is provided a 
method of enhancing gathered information comprising the steps of: 
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uploading at least one template comprising a plurality of template instructions; 
receiving at least one form of content information; 

executing the template instructions to generate composition instructions based 
on the captured content information; and 
5 enhancing a composition of the~conte~nt~ information in accordance "with tITe 

composition instructions. 

Preferably, the user interface means include buttons for a user to select a 
desired selection option and the composition instructions are relayed to the user through 
a display panel device. Alternately, at least some of the composition instruction are in 
10 the form of machine code which can directly or indirectly control hardware (e.g. 
content collecting unit such as charged couple devices (CCDs), transducers etc.) to 
affect the composition of the gathered content information. 

In accordance with a third aspect of the invention, there is provided a system 
for enhancing composition of gathered content information, the system including: 
15 content collecting means for gathering content information; 

template storage means for storing at least one template including a plurality of 
Template instructions; 

processing means for im.plementing the template instructions; and 

a user interface for displaying information to a user of the apparatus, wherein 
20 the template instructions when executed provide composition suggestions to a user of 
the content gathering apparatus in relation to enhancing composition of the content 
information collected, the composition suggestions being provided to the user via the 
user interface. 

Preferably, the suggestions are generated at least in part on the basis of ^he 
25 content information gathered. 

In accordance with a fourth aspect of the present invention^ there is provided a 
computer program product having a computer readable medium having a computer 
program recorded thereon for enhancing gathered information, the computer program 
product comprising: 

3o means for uploading at least one template comprising a pluralit>' of template 

instructions; 

means for receiving at least one form of content information; 

means for executing the template instructions to generate composition 
instructions based on the received content information; and 
35 means for enhancing a composition of the content infonnarion in accordance 

with the composition instructions. 



Brief Description of the Drawings 

Preferred embodiments of the present invention will now be described, by way 
of example only, with reference to the accompanying drawings, in which, in which: 

Fig. I is a block diagram representing an overview of an enhanced gathering 
device according to the embodiments of the present invention; 

Fig. 2 is a representation of digital still camera, comprising the enhanced 
gathering device of Fig. 1 in accordance with a first embodiment; 

Fig- 3 illustrates- a -Template Selection" mode of- the digital- still -camera of - 

Fig. 2; 

Fig. 4 illustrates a mode of the digital still camera of Fig. 2, in which 
composition features, "Orientation Guidance" and "Alignment" of an image to be 
gathered are shown; 

Fig. 5 illustrates an alternative mode for the mode of the digital still camera of 
Fig. 2, showing another composition feature and "Zoom Guidance"; 

Fig. 6 illustrates an example of an enhanced image composition in accordance 
with the present invention; 

Fig. 7 is an example of a viewfinder display or a digitaJ camera such as that 
shown in Fig. 2, for a portrait of two subjects in an image to be captured: 

Fig. 8 is a block diagram of an alternate embodiment of the present invention: 

Fig. 9 is an example of a check list associated with a template; 

Fig. 10 is a block diagram of information How in accordance with the present 
invention; and 

Fig. il is a flow diagram of a process of enhancing gathered content 
information, in accordance with the invention. 

Detailed Description 

Referring to Fig. 1, there is shown a block diagram setting out the 
relationships between major components of preferred embodiments of a content 
gathering device 100. The components include a content collection unit 101, which is 
operatively connected with a processing unit 104 and an output interface 102. The 
processing unit 104 is also connected with a memory storage unit 103, a user interface 
unit 105, a read only memory (ROM) 108 and a communications port 106. the 
communications port also has access to data contained within the memory unit 103. 

In alternative embodiments, the communications pon 106 can be omitted, and 
the memory unit 103 can be replaced with technology such as PCMCIA. 

The content collection unit 101 comprises one or more content sensors, each of 
which is capable of collecting or gathering predetermined forms of content information. 
Examples of such sensors include a microphone for receiving audio dau, a charge 
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coupled device (CCD) for collecting image data, and a light sensor for measuring 
lighting conditions associated with collected image data. 

Preferably, the content collection unit 101 further includes transducers to 
gather sensory information about a current state of the content sensors, or associated 
5 hardware such as the current zoom factor on a video camera. This additional data is 
available for use by the processing unit 104 and can be encoded as pan of metadata 

associated wiLh_ the gathered_cj)ntentJnfoiinatjon._ The associated metadata can include. 

such information as the date, time and location at which the image data was recorded, 
and the settings of the camera. 
'0 The gathered content information can be output via the output interface 102, 

for use by external devices. Such external devices can include content recording 
devices such as analogue or digital video recorders, optical disk drives, sound 
recording devices, image recording devices, primer devices or general purpose 
computers (such as an IBM-compatible personal compuier, an Apple-compatible 
15 personal computer or a Sun Workstation). 

A user interface device 105 comprising a liquid crystal display (LCD) conveys 
information such as messages, checklists and menus to a user, and a series of control 
buttons (not shown) accept user input in response thereto. The control buttons can be 
implemented on a touch-sensitive LCD. 
20 In use, the processing unit 104, in conjunction v/ith the memory storage unit 

103 and/or the ROM 108, provides the user with a selection of currently available 
templates via the user interface unit 105. The user initially uses selection buttons (not 
show n.> to select at least one of the templates. 

Each template comprises a series of template instructions which can be 
25 described by a predetermined programming language. For example, the programming 
language may be specific to an application and proprietary to the device 100 or can be 
an implementation of a general purpose programming language such as Java (Trade 
Mark of Sun Microsystems Inc.). It will be appreciated by those skilled in the art, 
however, that the template can take any suitable form. For example, the template can 
30 take the form of an execuuble file, a script file, an expen system or database, or any 
other type of structure. Also, the template can include multiple components and sub- 
components, each of which can take any suitable form. In this way, a template can 
comprise a script file in combination with, say. an expert system or any other desired 
combination. 

35 In the embodiment shown, templates are uploaded from a general -purpose 

computer system 107 via the communications pon 106 to the enhanced gathering device 
100, and are stored in the memory storage unit 103. 

In an alternative embodiment, one or more of the templates are stored on a 
cartridge, fiashcard, magnetic or optical disc, or any other type of removable storage 
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medium, which can simply be plugged into a suitable receptacle in the gathering device 
100. 

In a preferred embodiment, some or all of the template instructions are stored 
in the ROM 108 to reduce the amount of RAM memory required in the storage unit 
103. It is panicularly desirable that the ROM store at least those template instructions 
which are common to several templates. The template instructions are then available to 
the processing unit 104 fbr runtime calculations. " 

Preferably, the general purpose computer system 107 illustrated in Fig. 1 is 
also used as the external device previously described, and is used to download and 
record the content information from the content gathering device 100 via output 
interface 102. 

The processing unit 104 interprets the template instructions, presents the user 
with various options from a series of menus in accordance with the interpreted 
instructions, and interprets user inputs. The choices selected by the user can affect the 
current state of the content collection unit lOI, user preference settings, and the current 
template being used by the content gathering device 100. 

Fig. II is a now diagram illustrating the general process of enhancing gathered 
information in accordance with the present invention. The process commences at step 
1 100. In step 1 102, at least one template is uploaded in the enhanced gathering device, 
the template usually being selected by the user. In step 1104, at least one form of 
content information is received by the gathering device. In step 1106, template 
instructions from the uploaded template are executed to generate composition 
instructions based on the received content information. In step 1108, the composition 
of the content information is enhanced in accordance with the composition instructions. 
In step 1110, processing terminates. Preferably, the method further involves the step of 
storing the enhanced content information in a storage means and optionally 
downloading to an external device: It is particularly desirable in step 1 108 that the 
content information is enhanced at least partly on the basis of the content information 
itself. By interpreting the content information itself, dynamic, customised enhancement 
is enabled. 

In a preferred embodiment, some of these tasks are automated. A decision on 
which content to gather, and with what composition, is made by the gathering device 
100 on the basis of the currently selected template. For example, in a specific first 
embodiment of the present invention described hereinafter, the template is used to 
decide from predetermined conditions that a portrait view of an image is preferred over 
a landscape view, and so cropping of the image io a portrait view is automatically 
performed if necessary. 

Other features and advantages of the present invention are set forth in greater 
detail hereinbelow. 
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First FtTibodiimffni 

Rererring .o Figs. 2 .o 7, .he pr«cn. invention can be impiememed in the fom, 
of a d,g„a, s,„i camera 202 having a gathering device 100 in accordance with Z 
s .hown ,n F,g. , built in. The content collecting unit 101 takes the fo™, of an optica 
.ens systen, and a^ociated CCD. as is typical for such can,eras. and can also include 
distance nteasurtng sensors, light intensity determination devices, focus detection 
devtces and position locating devices (e.g. a Global Positioning Systen, (GPS) 
_ -apparatus, _These. sensors -and devices gather in,age-da,a.-and metadata- such a's the " 
3 location where the photograph was taken. 

The user interface unit 105 for the dig.tal still camera 202 takes the form of a 
Mu,d ctysta. display (LCD) 203. which is touch sensitive to enable user input 
Alternatively other data input means, such as buttons, can be used. 

Fig. 2 is a schematic representation of the digital still camera 202 used in 
taking a picture of subject matter 201. The LCD 203 associated with the digital still 
camera 202 is used to display an image to be recorded. In use. the user can record an 
.mage by pressing a shutter button 206 or can enter a template mode bv pressing a 
n^ode template button 204. The user can e.xit the template mode by pressing the m^e 
template button 204 a second time. 

Turning now to Fig. 3. initially pressing the mode template button 204 causes 
a list Of available templates 301 to be displayed on the LCD 203. In this example the 
templates include a landscape template, a pan shot template and a ponrait template 
which were uploaded to the gathering device as described above. A selector button 205 
•s used to navigate through the available options, and to select the desired template A 
selection arrow 302 displayed on the LCD 203 indicates the current template selection 

Once a template is selected, the image displayed on the LCD 203 is 
periodically or continually analysed by the processing unit 104. The results of the 
analysis are used by the template to guide the user and make suggestions in relation to 
composition. In the example shown, the portrait template requires that for best results 
the subject matter 201 should be generally centred and occupy at least a predetermined 
percentage of viewing field to be captured. 

Turning to Fig. 4. once a template has been selected, the LCD 203 shows a 
number of elements, including a positioning indicator 401. a line up marker 404 and a 
centre marker 405. fhese elements, although visible on the LCD 203. are not captured 
as part of any image when che user depresses the shutter button 206. Depending upon 
the implementation, the elements can appear on a portion of the display in whfchl 
:mage IS not displayed, or can be composited over the image. ,n the latter case it 
ZTZyZslT^''' " ^^^'^"^ ~ - - - i's not 
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The current sums of the positioning indicator 401, the line up marker 404 and 
the centre marker 405 are associated and stored with the coricsponding image . as 
metadata for later use or evaluation. For example, when capturing the image, the user 
might not correctly align one or more of the markers 404, 405 or positioning indicator 
401. if the content collecting unit is capable of gathering a larger view of an image 
than suggested by the markers and positioning indicator, then the information pan of 
the metadata can be used to suggest a preferred print area for the image during 
subsequent viewing and processing of the image. 

The" position indicator 401 provides suggested -zoom information -to the user. 
A "zoom in" arrow 403 indicates that it is considered desirable to zoom in on the 
subject 201 of the image to improve the composition of the proposed phorograph. 
Correspondingly, a "zoom out" arrow 406 can flash to indicate that it is desirable to 
zoom out from the subject 201 to improve the composition. Alternatively, the subject 
201 or the camera 202 can be moved closer together or farther apart to achieve an 
effect substantially similar to zooming. 

The position indicator 401 can also advise the user to orient the camera 202 
vertically by flashing a peripheral arrow 402 on the indicator. As illustrated in Fig. 5. 
the user rotates the camera 202 into the suggested vertical orientation so that the 
peripheral arrow 402 no longer flashes, thereby indicating a correct orientation for 
improved composition. At this stage, as illustrated in Fig. 5, the markers 404. 405 are 
still not aligned, and the "zoom in" arrow 403 is indicating that a zoom in is desired. 

The centre marker 405 and the line up marker 404 are provided for a variation 
on the first embodiment in which the enhanced capture device 100 is capable of 
analysing the content information of the image and substantially detecting an 
approximate position of the subject 201 in the image. The detection of the approximate 
position of the subject of the image need not be performed by analysing the content 
information and can be determined by other techniques known to those skilled in the an 
without depaning from the scope or spirit of the invention. For example, the position 
of the subject in the image can be determined by detecting which ponion of the image a 
user is looking at from a relative position of the user's eye pupils in relation to the 
image being observed. 

In this variation of the first embodiment, the centre marker 405 is placed at the 
centre of the visible area, while the line up marker 404 is placed at the centre of a 
detected subject. Since the line up marker attempts to follow the subject 201. a user 
can manipulate the line up marker 404 and centre marker 405 by moving the camera 
202. 

The various position indicators and markers need not take the form of an "x", 
as is shown in the present embodiment. Depending upon the embodiment, any suitably 
shaped marker can be used. In one preferred alternative embodiment, one or more of 
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ihe indicators or markers can take the form of a circle or oval which roughly 
corresponds with the shape of the subject (eg. a face) to be captured. In this way. the 
user can move, zoom and change the orientation of the camera until the subject is 
positioned within the circle or oval, thereby to improve the composition of the resultant 
5 photograph. 

Tumine to Fig. 6, the user has centred and zoomed in on the subject matter 
201 of the image correctly, and all of the arrows on the position indicator have ceased 
flashing. For further convenience, in the embodiment shown, there is provided an 

indicator 602,of alignment on the LCD 203. TTie posinon indicator having no arrows 

[o displayed (and, in the preferred embodiment, by the indicator 602 being on) the 
composition of the image is considered by the camera 202 to correspond with that 
suegested by the selected template. The user can then activate the shutter button 206 to 
record the image. 

Fie. 7 shows the digital still camera 202 operating in portrait template mode, 

15 in which a plurality of objects 704 and 705 are to be captured. The template attempts 
to optimise a best fit of the objects 704. 705 into a visual area of the image. 
Preferably, a izeometric centre of a bounding box containing the objects is used as an 
aliizn marker 702 location, and a geometric centre of the visual area of the im^ge (i.e. 
the imaee visible in LCD 203) is used as a centre marker 701 . The object 70: is partly 

20 outside the visible display area 203, and so the position indicator recommends a zoom 
out by tlashing the zoom out arrow 406. As with the previous siiLaiion. th^r user 
adjusts the zoom, orientation and other composition aspects until the template's 
sussestions have been satisfied. The photograph can then be taken. 

A substantially similar description applies to the landscape and pan shot 

25 templates. The landscape template is generally used for photographing landscape 
scenery and the pan shot for panoramic v/ide view shots. 

For the sake of conciseness, the templates have been described as providing 
only relatively limited numbers and types of composition suggestions, such as 
orientation and zoom. However, it will be appreciated that the invention is not limited 

30 to these particular aspects of image composition. Other aspects include suggesting 
liehting arrangements, best angle for lighting in a photographic studio setting, and 
camera focus advice. The camera focus advice can be by way of suggesting camera 
settings to achieve a foreground focus and background defocus. or vice versa. It is 
preferred that these suggestions are made dynamically on the basis of the actual content 

35 information being viewed. 

The templates attempt to provide advice to the user, from the content 
information available, which could corresporxl with that which might be given by o:je 
skilled in the art of photography. Accordingly, the preferred embodiment provides 
suggestions on camera orientation, zooming and positioning of subject matiei to provide 
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guidance to ihe amateur photographer who does not have the skill:> necessary to 
produce a quality photograph, or a photograph of a predetermined style. 

The templates are implemented in code with the use of a programming 
language as previously described. Pseudocode for a portrait template is included in the 
accompanying Appendix A. The main routine of the pseudocode executes and runs 
while the digital still camera 202 is powered up, and analyses the image formed on die 
LCD 203 in the manner described above 

As described previously in relation to Fig. 3, when the user has pressed the 
template mode selection button the available templates are displayed. The user 
can select a desired" templater with the~sele^ and then exit the template 

selection mode. When the selection has been made, a software routine such as that 
represented by the pseudocode contained in Appendix A checks for various conditions 
that are required for zoom and orientation of the image in the viewfinder. 

The subroutine SetUpForTemplate returns a TRUE boolean value when the 
image is aligned and the main program "Void Mainf)" will flag that the image is 
aligned. 

As previously described, additional information such as metadata can be 
recorded and asso<:iated with an image. Examples of this additional information include 
date, time, actual zoom used, shutter <:peed. aperture, that the still is a portrait of a 
person, the number of people in the shot and the centre of each of their faces, as well as 
the geometric centre of the faces as a group. 

Second Embr-'diment 

A second embodiment of the present invention is similar to the first 
embodiment, except that the gathering system 100 forms pan of a video camera. 
Templates are provided for giving advice on features including zoom. pan. composition 
and directing (in the sense of anistic, aesthetic or compositional film direction) of a 
video clip. For example, a plurality of uploadable templates can be provided to a user 
when purchasing the camera, including templates for directing a wedding scene, a real 
estate agent's home selling promotional clip, and a country holiday scene which may 
include suggestions as to sights and wonders of the world that are recommended for 
video recording. For instance, the user taking a trip to Egypt can upload a template of 
preferred sights of North Africa, or Egypt specifically, to the memory storage unit 103. 
Besides p>erform;ng the other functions herein described, the template may further 
suggest recommended sights and places of interest which are considered worthy of 
videotaping. In the Egyptian example, one suggestion might be the Pyramids or even 
particular pyramids and specific a.spects thereof. 

As a further example, reference is made to an uploadable "wedding" template 
for directing videotaping of a wedding. A user intending to make a video recording of 
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a relative's wedding can upload the wedding lemplaie from the general purpose 
computer 107 to the video camera (not shown) via the communications port 106. The 
wedding template is stored as compiled code in the memory storage means 103, while 
features common to the wedding and other uploadablc templates form pan of the ROM 
108. Common features m.ay include derails of maximum and minimum zoom settings 
available for the particular video camera, aperture sellings possible and other features 
that may be common to a majority of templates for the video camera. 

When the user starts to record a wedding .scene using the video recorder, the 
wedding template is activated and advises the user as to the particular shots should be 
taken, and in which order, to produce a video clip which would correspi^nd (after 
editing, if n^ce.-^sary) with a relatively professional video recording. 

Preferably, the prompts are given chronoU)gicall\ I-or example, when using 
tlie weddmg template, the user is prompted lo take a slu)t of the groon*. entering the 
church, the groom standing at tlie alter, the bride leaving the car and iniiking her way 
up the steps ol the church, the bride walking down the aisle and s*i on l-urther. the 
user is also advised on more technical matters ol video coiMposi'inn. such as zonniine. 
panning, lighting and audio. I-or example, if the user /ixmis io<» quickly when 
recording :hc bride w;;lking down the aisle, a "slow down ' (ruliciior i\ flashed in the 
viewllnilcr ttf the videt> camera. Also, in specific scenes such .is this, tlie video camera 
can attempt to reM»giuse what is being recordeii and pro\ idc .ip.pr* tpriaie dynar.^ic 
coinpositionai suugesiions f-or example, wherj ihr hridc is walking doun the aisle, the 
system can attempt to recognise the aisle by nieans t)t camera luovcment and (he 
CMNtencc ol parallel lines representing pews Ihe bride can also \k rccogtnscd (as 
being ll:: l>iidel and suggestii»ns made as lo the nuisi apf>rttpriaic use «»i t!)c cair.era. 
including panning and /t>oming. given tfie user s proximity and posinon unfj respect to 
I he bride 

Preferably, the user uses a series )f menu based cortunands displayed on a 
vicwtlnder icquivalcnl to LCI) 20.^^ in the first cmUKJiment* to confirm which sccti<>n 
i»r portion the user is curretitly being recorded. The user uses control buttons to 
indicate the section that is atK>ui to be. or has just been, recorded Tor example, in ific 
wc-dding template, the user is given a display of a number of scciii>ns. or poriiims. 
which are suggested for recording, including an 'Aisle" secuon for when the bride is 
walking down the aisle. The user uses control buttons or a touch screen to indicate thai 
the aisle section is about to be recordetl (or has finished being recorded) This 
indication is recorded as metadata for the correspond itig portion of the video clip. 

P'or example, information designating a section of the videotaped recording of 
l\\c wedding scene as 'bride walks down aisle' is recorded as pan of the mct;;data for 
the corresponding scene. When editing the recorded wedding scene, preferably using 
an exicrrnl device such as the general computer 107 including h^irdv^are a software 
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video-editing components, the metadata associated with the videotaped wedding can be 
used to identify various scenes within the wedding. 

Third Embodiment 

A third embodiment of the present invention is a sound mining desk 
incorporating a content gathering device. A sound mixing desk is typically used for 
- gathering musical content-from a number of different channels", applying various effects 
and filters to individual r.nd/or groups of channels, then "mixing" the channels into one 
or more output channels that are recorded or used to drive amplifiers for transmission 
to an audience. 

For simplicity, a relatively simple two-channel interface sound mixing desk is 
described. The mixing desk is implemented using a computer to display controls, 
analyse content, provide user feedback, and to con!.rol filters, analysers, effects units 
and other hardware items. 

Referring to Fig. 8. there is shown a sound mixing desk system 800 
comprising: a plurality of content gathering sensors in the form of sound transducers 
801; a gathering device 802 into which signals from the transducers 801 are fed in two 
channels; a plurality of effects units, filter banks and analysers 810 connected to the 
gathering device 802 by data link 803; an output amplifier 809 with at least one output 
811; playbac: devices 81z. including a Compact Disc player, reel to reel tape or 
cassette tape unit connected to the gathering device 803; and a general -purpose 
computer system 804 for operating and controlling the other components. 

Information gathered by the audio transducers 801 and the other peripheral 
devices 812 is passed to the .-enhanced ^.*apture device 802, which processes the gathered 
information according to a selected template. One or more templates of interest can \rK' 
uploaded to the enhanced gathering device 802 via a data link 805, for storage therein. 

The enhanced gathering device 802 comprises substantially the same 
components as illustrated in Fig. 1, although it is configured for use with audio data 
from tlie audio transducers 801, rather than with video or still image data. In the 
embodiment of the sound mixing desk 800 shown, the user interface unit of Fig. I is 
not a separate unit, and instead is implemented on the general-purpose computer system 
804. 

The information gathered by the sensors 801 and other peripheral devices 802 
forms part of the content information. The content information preferably includes 
metadata such as time, date, recording duration length, musical style and title of track, 
which can be added to the content information in accordance with instructions by a 
selected template. 

Although a plurality of templates will usually be uploaded to the enhanced 
gathering device 802, a single template selection can be made by manipulation of a 
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mouse 806 forming part of the computer 804, in conjunction with an external display 
813 and keyboard 814 as necessary. In the preferred form, the selection is made using 
a menu or icon driven display. 

The content information gathered is passed via a communication link 803 from 
5 the enhanced gathering device 802 through the plurality of effects units, niter banks and 
analysers 810, to the output amplifier 809 and output 811. 

Content information is passed to the computer 804 through the data link 805, 
and data about the content information is displayed for the user on the display device 
813. Preferably, the content information is further analysed by a central processing 

10 unit which extracts data to be displayed. Alternatively, the content information is 
analysed by the enhanced gathering device and only relevant display data is transmitted 
via link 805 to the computer 804. 

The user can adjust various settings through manipulation of the mouse 806 
and keyboard 814. The various settings are conveyed on a bi-directional link 807 as 

15 control signals to alter parameters of the amplifier 809 and the plurality of effects units, 
filler banks and analysers 810 to correspondingly produce desired modifications to 
signal at the output 811. The bi-directional link 807 preferably also carries signals 
generated by a number of devices, including the plurality of effects units, filter banks, 
analysers 810, the output amplifier 809 and the playback devices 812, relating to 

20 current status and/or playback content information of the varit)us playback devices 812 
if required. 

A user can alter preference and default settings thri)ugh the computer system 
804. The default settings are generally dictated by the type and/or combination of 
components connected to the sound mixing desk. For instance, different choices of 

25 audio transducers or playback devices may require different default settings. 
Preference settings generally provide a manner of customising a sound mixing system 
to user preferred settings. Both preference and default settings are stored in non- 
volatile memory forming pan of the computer system 804. 

An example of an uploadable template is described below with reference to a 

30 musical group mixing template. Such a template provides advice to the u.scr in relation 
to mixing (and, if desired, recording) the sounds of a particular type of musical groups, 
sucli as a string quartet, a rock and roll band, a heavy metal band and a classical 
orchestra. Upon selecting and executing the appropriate musical group template, the 
user is prompted to select a music style and nominate a venue size (i.e small hail, 

35 outdoors, studio etc.). Preferably, the user is further prompted to identify which 
channels (or associated sensors) are to used to gather information (sound signals) for 
instrumenii and which sensors are for vocals. The musical group mixing template then 
produces a recommended list that provides best allocation of available audio traasducers 
(i.e., micrrpi^ones) for each channel. 
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Further, the user may select a "testing" option, in which the musical group 
mixing template analyses the content information and advises whether any of the 
sensors are picking up undesirable reflections from surrounding objects or surfaces, or 
are being driven into overload. If such problems are detected, the template can either 
nominate a solution (e.g. to re-allocate or reposition the microphones) or attempt to 
electronically compensat_e_for„the^distortion through the, effects and -filters available.- 
Similarly. the musical group can do a test performance, during which a sound level 
setting is determined for use as guide during the actual performance. 

As data is collected by the sensors 801. the enhanced gathering device 802 
extracts content information and provides it to the computer S04. The computer 804 
generates and sends compensation control signals in relation to the filters and effects on 
each channel. 

For example, a user can select a "heavy metal" music template with a high 
distortion level to the vocal channel. Boosting of the top end frequencies is suggested 
to the user, and is preferably implemented automatically in the absence of u.ser 
override. 

However, if the user selects a "classical" music template, then all the channels 
are bia.sed to a relatively "Mat" spectral setting. That is. only minimal cITccts arc added 
to the channels through the effect units or filler banks, so that (he instrument sounds are 
reproduced as faiihfully possible. 

As previously described, each template provides advice and suggesiioas to the 
u.ser which could otherwise be provided by one skilled in the art. In the present 
example the art is the art of sound mixing, and so the advice provided can relate to. f(ir 
example, echo, sound reflection, reverberation, tempo, relative channel amplitudes, 
panning and pa/ametric or graphic frequency manipulation. 

Whilst so far only briefly described in relation to the second embodiment, an 
optional preferred feature of the templates applicable to each of the above de.scribed 
embodiments is that of providing a check list of events or actions that a user can "tick 
off in respect of predetermined events or actions. For instance, in the digital still 
camera example described above, a check list of predetermined shots is provided to a 
user through the user interface. Before (or after, depending upon the implementation) 
taking a particular photograph, the user ticks off a corresponding shot from the check 
list. For example, the digital still camera can execute a birthday party template which 
includes a birtliday party check list. The check list is displayed on the user interface 
and can include the following suggested photograph opportunities: 
*■ birthday pany games; 
* blowing out of the candles; and 
*■ making a toast to the birthday person. 
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The user takes photographs during the birthday parly. If a particular 
photograph taken coincides with an entry in the check list, that entry is "ticked off" by 
the user selecting it via the control buttons. Upon selecting the entry in the check list, 
metadata is generated and associated with the photograph. This metadata information 
can be utilised, for example, when the photograph (or image) is downloaded to a 
general purpose computer, to put together a photo-album of the birthday party. 
Software running on^ the computer extracts the Captured photographs, along with any 
associated metadata, and uses the metadata to arrange the photographs in the correct 
order and context to automatically generate a "virtual" photo album. The virtual photo 
album can include such additional features as appropriate music, borders, graphic art 
and text relevant to each of the photographs. 

A plurality of check lists can be provided with each template to assist the user 
in determining which photographic shots would provide good composition when placed 
adjacent each other in a photographic album . 

In the example of the sound mixing desk, a substantially similar check list for 
music can be provided upon which the user can indicate when a particular event has 
been captured or gathered. For example, the check list could be for piece ; of music, or 
can relate to checking particular mixing characteristics. Again, corresponding metadata 
is ucneraied and associated with the event. The corresponding metadata can be a 
simple identifier for the event or a more complex description of the event as may be 
desired for future manipulation or editing thereof. 

As well as providing template instructions which generate composition 
preferences based on the gathered content information, the template can contain 
additional information such as the order in which to implement or make each 
composition suggestion, the encoding or compression technique to be used with the 
content information, and which metadata is to be recorded. 

Referring now to Fig 9, there is shown another example of a check list 
displayed on the user interface display (e.g. viewfmder 203 of the digital still camera of 
Fig 2 or the viewfinder of the video camera of the second embodiment hereinbefore 
described). The check list 900 is associated with a real estate template which is 
uploaded into the enhanced gathering device 100 as previously described with reference 
to the first or second embodiment of the present invention. The real estate template in 
this case makes suggestions to the user (typically a real estate salesperson) regarding 
the suggested images required to compose a good presentation. The real estate template 
can, as an optional function, provide a check list comprising: description fields 901 
which provide a brief description of a recommended photograph (or video clip in the 
case of the video camera) to be recorded; check list indicator fields 902 provided for 
marking off each corresponding image (photograph) recorded; and check list marker 
903 to indicate which of the recommended images have been recorded. 



- 16 - 



The check list marker 903 is typically added by the user, through an available 
user input interface, preferably when the recommended image has been recorded. The 
real estate template, upon checking off the items of the check list, generates metadata 
and associates the metadata with the corresponding image of the check list. Preferably, 
the metadata is an encoded description of the image to which it is associated. Further, 
the nietadafa^cari include such"characteristics 'as"timerda^^^ - - - 

Referring to Fig. 10, there is illustrated a block diagram of information flow 
comprising uploadable templates 1002 stored on a computer system 1001. The 
iiploadable templates 1002 are uploadable from the computer system 1001 via a bi- 
directional communication link 1005 to a content gathering device in the form of input 
device 1000. The templates are stored as uploaded templates 1004 in the input device. 

Content information 1006 is gathered by the input device 1000, and metadata 
1007 is generated by at least one of the uploaded templates 1004. The generated 
metadata 1007 can simply be a descripfion of the content information determined either 
by a user response to a check lisc or from the template by analysis of the content 
information 1006. 

The template associates the generated metadata 1007 with a corresponding part 
or all of the content information 1006. For example, in the case of the real estate 
template, the content information can comprise an image from the check list 900 of 
images suggested by the real estate template. The corresponding metadata generated 
for the image can be based upon features including: content analysis directed to 
orientation, average colour, average brightness, dominant colours, predetermined 
patterns, and statistical analysis in spatial, time or frequency domains identified in the 
content information; user preferences including colour and arrangement of information; 
and device specific features including a current state of hardware components and 
predetermined formatting of information for downloading to various output devices. 

The gathered information, which includes content information and m.etadata, is 
downloaded to an external device. In the present example, the external device is the 
general purpose computer system 1001 as previously described. 

Alternatively, the external device can be a dedicated external device capable of 
receiving the gathered information. Such dedicated devices include: a printer device, a 
display device (e.g. cathode ray tube, CTR or Liquid Crystal Display (LCD)), a 
photocopying device, or an acoustic recording device. In each case, the dedicated 
device is configured to receive the gathered information and provide an output, or to 
store the information, according to predetermined preferences. Without departing from 
the scope or spirit of the present invention, the external device can be, for example, a 
video cassette tape for recording and storage of the gathered information to be used at a 
latter stage in composing a final output. 
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Preferably, an uploaded template 1004 formats and arranges the gathered 
information in accordance with predetermined settings, or formats the gathered 
information for compatibility with the type of external device to which it is to be 
downloaded. 

The process of enhancing gathered information in accordance with the 
embodiments of the invention can be implemented in software, readable and executable 
by a processing device such as a microprocessor, and recorded on a computer readable 
recording medium. The recording medium can be magnetic or optical storage memory 
such as a hard or floppy drives, magnetic tape, CD-ROM, rnagneto-optical disk, a" 
memory card, read-only memory, or any other type of storage device with sufficient 
capacity. Also, the software can reside on another computer system accessed through a 
computer network or across the Internet. 

Although the invention has been described with reference to a number of 
particular embodiments, it will be appreciated by those skilled in the art that the 
invention can be embodied in many other forms without departing from the scope or 
spirit of the invention. 
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Appendix 

Psuedo Code 

5 // Ponrait Template 

Boolean SetUpForPonrait(Boolean UI_Flags[]. image) 
{ 

Boolean Result = TRUE 

// "Assume only-caiied IF 0 < numFacesDetected < 4 

> IF IndequateMarginAroundGeometricCentre(image) THEN 

UI_FIags[ZOOM_OUT] = TRUE 
Result = FALSE 

ELSEIF ToMuchMarginAroundGeometricCentre(image) THEN 
UIFIagsfZOOMJNl = TRUE 
Result = FALSE 

ENDIF 

IF ((FacesBoundingBoxTa!lerThanJtIsWide(image) AND 
CameraOrientation Equals LANDSCAPE) OR 

(FacesBoundingBoxWidererThanItIsTaII(imacc> AND 
CameraOrientation Equals PORTRAIT)) Then 

UI FlagsfCHANGE ORIENTATIONl = TRliE 
Result = FALSE 

ENDIF 

IF (FacesBoundingBoxCentrcd(image) EQUALS FALSE THEN 

UI_FLAGS[CENTRE_B0UNDING30XI = TRUE 
Result = FALSE 

ENDIF 

RETURN Result 

} 



Void mainO 

Boolean UIFlags(CONST_NUM UI_FLAGS) 
35 ImageType image 

TemplateType templateSelected 

WHILE PoweredOn 

IF McnuButtonPiessedO Then 
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tempIateSelected = DisplayAvailableTempIatesO 

ENDIF 

// Get the image that the camera is looking at 
Ge timage InViewrmder(image) 

// Check for cases for which there is a template 

IF tempIateSelected EQUALS CONST_PORTRAT_TEMPLATE THEN 
IF 'SeiUpForPortrait(UIFrags . imag"e)"EQU AIlS TRUE THEN 

UIFlagslPICTURE_ALIGNEDl = TRUE 

ENDIF 

ELSEIF 

Other checks for other templates 

ENDIF 

UpdateUI(UIFhigs) 
END WHILE 
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The claims defining the invention are as follows: 

1. A content gathering apparatus including: 
content collecting means for gathering content information; 

template storage means for storing at least one template including a plurality of 
template instructions; 

processing means for implementing the template insiructions; and 
a user interface Tor displaying information to a user of the apparatus, wherein 
the template instructions when executed provide composition suggestions to a user of 
the content gathering apparatus in relation to enhancing composition of the content 
information collected, the suggestions being provided via the user interface. 

2. A content gathering apparatus according to claim 1, wherein the 
suggestions are generated at least in pan on tht basis of the content informatic^n 
gathered. 

3. A content gathering apparatus according to claim I or 2, wherein the 
template instructions, when executed, further provide composition instructions 
corresponding to the composition suggestions, for implementation by the content 
gathering apparatus in the absence of countermanding instructions from the user. 

4. A content gathering apparatus according to any one of the preceding 
claims further including content information storage means for siorine the gathered 
content informaticin. 

5. A content gathering apparatus according any one of the preceding 
claims, funher including communication means for uploading the template to the 
gathering apparatus 

6. A content gathering apparatus according to any one of the preceding 
claims, wherein the template instructions are coded machine insiructions for the content 
collecting means. 

7. A content gathering apparatus according to claim 6. wherein the 
content collecting means receives said coded machine instructions and accordingly 
alters at least the composition of the gathered content information. 



- 21 - 



8. A content gathering apparatus according to any one of the preceding 
claims, wherein the user interface includes input means for allowing user input in 
response to display of said composition suggestions 

9. A content gathering apparatus according to any one of the preceding 
claimsr wherein the gathered content information includes.meiadai_a, 

10. A content gathering apparatus according to claim 9, wherein the 
metadata is selected from the group consisting of the following pieces of informacion in 
relation to the content information gathered: 

date and time; 
location; 

camera orientation information; 
camera setting information; and 
information on user selected options 

11. A content gathering apparatus according to any one of the preceding 
claims, the content gathering apparatus being selected from the group consisting ol: 

a still camera; 

a video recorder: 

an image capture device; 

an audio recorder; 

an audio mixer; and 

an audio capture device. 

12. A method of gathering content information using a content gathering 
apparatus, including the steps of: 

providing at least one template to the content gathering apparatus, the template 
including a plurality of template instructions; 

capturing at least one form of content information with the content gathering 

apparatus; 

executing the template instruciioas to generate composition instructions based 
on the captured content information; and 

enhancing composition of said content information in accordance with the 
composition iasiruciions. 

1,^. A method according to claim 12, further including the step of storing 
the enhanced content information in a storage meaas associated with the content 
gathering apparatus. 
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14. A method according to claim 12 or 13, further including the step of 
downloading the composition enhanced content information to an external device. 

15. An apparatus including a readable storage medium having a computer 
program recorded therein for enhancing gathered content information, said apparatus 
including: 

"niearis for uploading at least one template from the readable storage medium,- 
the template including a plurality of template instructions; 

means for gathering at least one form of content information; 

a computer pr(x:essor for executing said template instructions, thereby to 
generate composition instructions based on the content information; and 

means for enhancing a composition of said content information in accordance 
with the composition instructions. 

16. An apparatus according to claim 15. further including means fi)r 
storing the enhanced content information on a storage means. 

17. An apparatus according to claim 16, further including mcan.s tor 
downloading the enhanced content information to an external device. 

18. A system for enhancing composition of gathered content miormarion. 
the system including: 

content collecting means for gathering content inlbrmaiion: 

template storage means for storing at least one template including a plurality of 
template instructions; 

processing means for implementing the template instructions; and 
a user interface for displaying information to a user of the apparatus, wherein 
the template instructions when executed provide composition suggestions to a user of 
the content gathering apparatus in relation to enhancing composition ot the content 
information collected, the composition suggestions being provided to the user via the 
user interface. 

19. A system according to claim 18. further including a computer for 
storing a plurality of templates for selective uploading to the template storage means. 

20. A system according to claim 19, wherein the content collecting means, 
template storage means, processing me<.-is and user interface are implemented on a 
content gathering apparatus. 
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21. A system according lo claim 20. the user interface including user 
input meanc tor allowing a user to input responses to the composition suggestions" 

22. A system according to claim 21 . wherein the computer is proerammed 
w.th software for manipulating the gathered content information on the basis of the 

^composition suggestions and user responses thereto". - - 

23. A software storage medium bearing content information which has 
been composition enhanced by means of the apparatus defined in anv one of claims 1 to 
1 1, or 15 to 17. 

24. A software storage medium bearing content information which has 
been composition enhanced by means of the method of any one of claims 12 to 14. 

25. A software storage medium beari -.g content information which has 
been composition enhanced by mean.s of the .-system of anv one of claims 

18 to 22. 

26. A content gathering apparatus .substantially as herein dcscril^-d with 
reference u, any one of the emb,>dimenis of the invention shown in the accompanying 
drawings. 

27. A methfxJ of gathering content information substaniiailv as herein 
described with reference to any one of the embcxJiments of the mvention shown in the 
accompanying drawings. 

28. A .system for gathering content information substantially as herein 
described with reference to any one of the embodiments of the invention shown in the 
accompanying drawinss. 
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ABSTRACT 

A content gathering apparatus (100) including content collecting means (101) tor 
gathering content information and template storage means (103) for storing at least one 
template including a plurality of template instructions. Processing means (104) are 
used to implement the template instructions. A user interface (105) displays 
information to a user of the apparatus (100). The template instructions, when executed, 
provide composition suggestions to the user of the content gathering apparatus (100) in 
relation to enhancing composition of the content information collected, the sugges;ions 
being provided via the user interface ( 105). Methcxls and systems for implementing 'he 
invention are also disclosed. 
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